Distributed resource allocation in wireless networks: a game-theoretical learning framework

Device-to-device (D2D) communications underlaying a cellular infrastructure is regarded as one of the key technology enablers for future wireless networks. The main advantages of underlay D2D communications stem from the reuse-, proximity-, and hop gains, which can be utilized for enhanced coverage, capacity and quality-of-service in mobile networks. The basic idea consists in enabling suitably-selected nearby device pairs to reuse the cellular spectrum for direct data transfer, while ensuring that there is no detrimental impact on traditional cellular transmissions via base stations. Despite its great potential for performance gains, D2D communications poses some fundamental challenges to system designers. These challenges, which include D2D discovery, transmission mode selection, resource allocation and interference management, are exacerbated by the lack of timely and accurate channel state information for direct D2D links at the level of base stations and wireless devices. Therefore, in order to avoid a significant increase in the feedback and signaling overhead, there is a strong need for D2D resource allocation solutions that (i) are amenable to distributed implementation and (ii) can beneficially exploit some sideinformation made available at the level of D2D links through the network assistance. In addition, such D2D solutions must be capable of dealing with the following characteristics of mobile networks: ∙ uncertainty, which is caused by the random nature of the wireless environment (including channels and users’ behavior), and is further aggravated by the lack of information at the user level, and ∙ competition between users that attempt to access strictly limited wireless resources. To address these challenges, the core objective of this thesis is to develop and study a novel theoretical framework for network-assisted D2D resource allocation that incorporates game theory and reinforcement learning. We model a distributed D2D wireless network as a multi-agent system, in which a set of self-interested smart agents with bounded rationality share limited resources, by taking actions according to some decision making strategy. Every joint action profile is associated with some reward (or cost) for each agent, and the agents selfishly compete for access to resources in order to achieve a higher utility. By incorporating a learning model into a game-theoretical formulation, the agents’ actions